Feature Extraction based on Space Folding Model and Application to Machine Learning

One of the most important designs for a lot of machine learning methods is the determination of the similarity between instances. Especially the kernel matrix, which is also known as the Gram matrix, plays a central role in the kernel machines such as support vector machine. The simplest design of similarity function is to use the distances between instances or the Gaussian function based on them. It is easy to learn the model when the data distribution follows their label, in which the instances with same label are allocated near and those with different label are allocated far. However, when the data distribution is non-linear, it becomes difficult. This paper discusses the inner products of 2 non-orthogonal basis vectors and proposes the similarity between instances. This paper also proposes a space folding model for machine learning based on the proposed similarity. This paper applies the proposed method to pattern recognition problem and shows its effectiveness.